SimpleDIVA: A 3-parameter model for examining adaptation in speech and voice production

Published versio

Representation of Sound Categories in Auditory Cortical Maps

We used functional magnetic resonance imaging (fMRI) to investigate the representation of sound categories in human auditory cortex. Experiment 1 investigated the representation of prototypical and non-prototypical examples of a vowel sound. Listening to prototypical examples of a vowel resulted in less auditory cortical activation than listening to nonprototypical examples. Experiments 2 and 3 investigated the effects of categorization training and discrimination training with novel non-speech sounds on auditory cortical representations. The two training tasks were shown to have opposite effects on the auditory cortical representation of sounds experienced during training: discrimination training led to an increase in the amount of activation caused by the training stimuli, whereas categorization training led to decreased activation. These results indicate that the brain efficiently shifts neural resources away from regions of acoustic space where discrimination between sounds is not behaviorally important (e.g., near the center of a sound category) and toward regions where accurate discrimination is needed. The results also provide a straightforward neural account of learned aspects of categorical perception: sounds from the center of a category are more difficult to discriminate from each other than sounds near category boundaries because they are represented by fewer cells in the auditory cortical areas.National Institute on Deafness and Other Communication Disorders (R01 DC02852

ROI-Based Analysis of Functional Imaging Data

In this technical report, we present fMRI analysis techniques that test functional hypotheses at the region of interest (ROI) level. An SPM-compatible Matlab toolbox has been developed which allows the creation of subject-specific ROI masks based on anatomical markers and the testing of functional hypotheses on the regional response using multivariate time-series analysis techniques. The combined application of subject-specific ROI definition and region-level functional analysis is shown to appropriately compensate for inter-subject anatomical variability, offering finer localization and increased sensitivity to task-related effects than standard techniques based on whole brain normalization and voxel or cluster-level functional analysis, while providing a more direct link between discrete brain region hypotheses and the statistical analyses used to test them.National Institute of Health (R29 DC02852, ROI DC02852

Changes in the McGurk Effect Across Phonetic Contexts

To investigate the process underlying audiovisual speech perception, the McGurk illusion was examined across a range of phonetic contexts. Two major changes were found. First, the frequency of illusory /g/ fusion percepts increased relative to the frequency of illusory /d/ fusion percepts as vowel context was shifted from /i/ to /a/ to /u/. This trend could not be explained by biases present in perception of the unimodal visual stimuli. However, the change found in the McGurk fusion effect across vowel environments did correspond systematically with changes in second format frequency patterns across contexts. Second, the order of consonants in illusory combination percepts was found to depend on syllable type. This may be due to differences occuring across syllable contexts in the timecourses of inputs from the two modalities as delaying the auditory track of a vowel-consonant stimulus resulted in a change in the order of consonants perceived. Taken together, these results suggest that the speech perception system either fuses audiovisual inputs into a visually compatible percept with a similar second formant pattern to that of the acoustic stimulus or interleaves the information from different modalities, at a phonemic or subphonemic level, based on their relative arrival times.National Institutes of Health (R01 DC02852

Brain-Computer Interfaces for Speech Communication

This paper briefly reviews current silent speech methodologies for normal and disabled individuals. Current techniques utilizing electromyographic (EMG) recordings of vocal tract movements are useful for physically healthy individuals but fail for tetraplegic individuals who do not have accurate voluntary control over the speech articulators. Alternative methods utilizing EMG from other body parts (e.g., hand, arm, or facial muscles) or electroencephalography (EEG) can provide capable silent communication to severely paralyzed users, though current interfaces are extremely slow relative to normal conversation rates and require constant attention to a computer screen that provides visual feedback and/or cueing. We present a novel approach to the problem of silent speech via an intracortical microelectrode brain computer interface (BCI) to predict intended speech information directly from the activity of neurons involved in speech production. The predicted speech is synthesized and acoustically fed back to the user with a delay under 50 ms. We demonstrate that the Neurotrophic Electrode used in the BCI is capable of providing useful neural recordings for over 4 years, a necessary property for BCIs that need to remain viable over the lifespan of the user. Other design considerations include neural decoding techniques based on previous research involving BCIs for computer cursor or robotic arm control via prediction of intended movement kinematics from motor cortical signals in monkeys and humans. Initial results from a study of continuous speech production with instantaneous acoustic feedback show the BCI user was able to improve his control over an artificial speech synthesizer both within and across recording sessions. The success of this initial trial validates the potential of the intracortical microelectrode-based approach for providing a speech prosthesis that can allow much more rapid communication rates

Auditory feedback control mechanisms do not contribute to cortical hyperactivity within the voice production network in adductor spasmodic dysphonia

Adductor spasmodic dysphonia (ADSD), the most common form of spasmodic dysphonia, is a debilitating voice disorder characterized by hyperactivity and muscle spasms in the vocal folds during speech. Prior neuroimaging studies have noted excessive brain activity during speech in ADSD participants compared to controls. Speech involves an auditory feedback control mechanism that generates motor commands aimed at eliminating disparities between desired and actual auditory signals. Thus, excessive neural activity in ADSD during speech may reflect, at least in part, increased engagement of the auditory feedback control mechanism as it attempts to correct vocal production errors detected through audition. To test this possibility, functional magnetic resonance imaging was used to identify differences between ADSD participants and age-matched controls in (i) brain activity when producing speech under different auditory feedback conditions, and (ii) resting state functional connectivity within the cortical network responsible for vocalization. The ADSD group had significantly higher activity than the control group during speech (compared to a silent baseline task) in three left-hemisphere cortical regions: ventral Rolandic (sensorimotor) cortex, anterior planum temporale, and posterior superior temporal gyrus/planum temporale. This was true for speech while auditory feedback was masked with noise as well as for speech with normal auditory feedback, indicating that the excess activity was not the result of auditory feedback control mechanisms attempting to correct for perceived voicing errors in ADSD. Furthermore, the ADSD group had significantly higher resting state functional connectivity between sensorimotor and auditory cortical regions within the left hemisphere as well as between the left and right hemispheres, consistent with the view that excessive motor activity frequently co-occurs with increased auditory cortical activity in individuals with ADSD.First author draf

Behavioral and neural correlates of speech motor sequence learning in stuttering and neurotypical speakers: an fMRI investigation

Stuttering is a neurodevelopmental disorder characterized by impaired production of coordinated articulatory movements needed for fluent speech. It is currently unknown whether these abnormal production characteristics reflect disruptions to brain mechanisms underlying the acquisition and/or execution of speech motor sequences. To dissociate learning and control processes, we used a motor sequence learning paradigm to examine the behavioral and neural correlates of learning to produce novel phoneme sequences in adults who stutter (AWS) and neurotypical controls. Participants intensively practiced producing pseudowords containing non-native consonant clusters (e.g., “GVAZF”) over two days. The behavioral results indicated that although the two experimental groups showed comparable learning trajectories, AWS performed significantly worse on the task prior to and after speech motor practice. Using functional magnetic resonance imaging (fMRI), the authors compared brain activity during articulation of the practiced words and a set of novel pseudowords (matched in phonetic complexity). FMRI analyses revealed no differences between AWS and controls in cortical or subcortical regions; both groups showed comparable increases in activation in left-lateralized brain areas implicated in phonological working memory and speech motor planning during production of the novel sequences compared to the practiced sequences. Moreover, activation in left-lateralized basal ganglia sites was negatively correlated with in-scanner mean disfluency in AWS. Collectively, these findings demonstrate that AWS exhibit no deficit in constructing new speech motor sequences but do show impaired execution of these sequences before and after they have been acquired and consolidated.Published versio

A Wireless Brain-Machine Interface for Real-Time Speech Synthesis

This is the published version, also available here: http://dx.doi.org/10.1371/journal.pone.0008218.Background Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech. Methodology/Principal Findings Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task. Conclusions/Significance Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas

Evaluating the validity of volume-based and surface-based brain image registration for developmental cognitive neuroscience studies in children 4 to 11 years of age

Understanding the neurophysiology of human cognitive development relies on methods that enable accurate comparison of structural and functional neuroimaging data across brains from people of different ages. A fundamental question is whether the substantial brain growth and related changes in brain morphology that occur in early childhood permit valid comparisons of brain structure and function across ages. Here we investigated whether valid comparisons can be made in children from ages 4 to 11, and whether there are differences in the use of volume-based versus surface-based registration approaches for aligning structural landmarks across these ages. Regions corresponding to the calcarine sulcus, central sulcus, and Sylvian fissure in both the hemispheres were manually labeled on T1-weighted structural magnetic resonance images from 31 children ranging in age from 4.2 to 11.2 years old. Quantitative measures of shape similarity and volumetric-overlap of these manually labeled regions were calculated when brains were aligned using a 12-parameter affine transform, SPM's nonlinear normalization, a diffeomorphic registration (ANTS), and FreeSurfer's surface-based registration. Registration error for normalization into a common reference framework across participants in this age range was lower than commonly used functional imaging resolutions. Surface-based registration provided significantly better alignment of cortical landmarks than volume-based registration. In addition, registering children's brains to a common space does not result in an age-associated bias between older and younger children, making it feasible to accurately compare structural properties and patterns of brain activation in children from ages 4 to 11

A Wireless Brain-Machine Interface for Real-Time Speech Synthesis

Background: Brain-machine interfaces (BMIs) involving electrodes implanted into the human cerebral cortex have recently been developed in an attempt to restore function to profoundly paralyzed individuals. Current BMIs for restoring communication can provide important capabilities via a typing process, but unfortunately they are only capable of slow communication rates. In the current study we use a novel approach to speech restoration in which we decode continuous auditory parameters for a real-time speech synthesizer from neuronal activity in motor cortex during attempted speech. Methodology/Principal Findings: Neural signals recorded by a Neurotrophic Electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome, characterized by near-total paralysis with spared cognition, were transmitted wirelessly across the scalp and used to drive a speech synthesizer. A Kalman filter-based decoder translated the neural signals generated during attempted speech into continuous parameters for controlling a synthesizer that provided immediate (within 50 ms) auditory feedback of the decoded sound. Accuracy of the volunteer's vowel productions with the synthesizer improved quickly with practice, with a 25% improvement in average hit rate (from 45% to 70%) and 46% decrease in average endpoint error from the first to the last block of a three-vowel task. Conclusions/Significance: Our results support the feasibility of neural prostheses that may have the potential to provide near-conversational synthetic speech output for individuals with severely impaired speech motor control. They also provide an initial glimpse into the functional properties of neurons in speech motor cortical areas.National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R44-DC007050)National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R01-DC007683)National Institute on Deafness and Other Communication Disorders (U.S.) (Grant R01-DC002852)Center of Excellence for Learning in Education, Science, and Technology (SBE-0354378